Method for generating human-computer interactive abstract image

ABSTRACT

A method for generating a human-computer interactive abstract image includes: S1: obtaining and preprocessing the original abstract images used as a training dataset B to obtain edge shape feature maps used as a training dataset A; S2: using the training dataset A and the training dataset B as cycle generative objects of a Cycle-GAN model, and training the Cycle-GAN model to capture a mapping relationship between the edge shape feature maps and the original abstract images; S3: obtaining a line shape image drawn by a user; and S4: according to the mapping relationship, intercepting a generative part in the Cycle-GAN model that the dataset B is generated from the dataset A, discarding a cycle generative part and a discrimination part in the Cycle-GAN model, and generating a complete abstract image based on the line shape image to generate the human-computer interactive abstract image.

CROSS REFERENCE TO THE RELATED APPLICATIONS

This application is based upon and claims priority to Chinese PatentApplication No. 202011417498.3, filed on Dec. 7, 2020, the entirecontents of which are incorporated herein by reference.

TECHNICAL FIELD

The present invention relates to the technical field of imagegeneration, and more particularly, to a method for generating ahuman-computer interactive abstract image.

BACKGROUND

As deep learning advances, the use of a generative adversarial network(GAN) enables artificial intelligence (AI) to apply to the field ofimage generation. In contrast to traditional image processing approachesbased on manual feature analysis, the deep learning enables computers toadaptively analyze latent features of given data. This is especiallysuitable for processing image data whose features are difficult toclearly define and classify, such as artworks. Based on the idea ofgenerative adversarial algorithm, public studies have explored networkstructures that can be used to generate artworks. As shown in FIG. 1 , agenerative structure includes a generator G_(θ)(•) and a discriminatorD_(ϕ)(•). Noise z conforming to a Gaussian distribution is input intothe generator G_(θ)(e) to obtain a generated artistic image G_(θ)(z),and then the artistic image G_(θ)(z) is mixed with a real artistic imageX_(real) to form a mixed image set. An image randomly selected from themixed image set is input into the discriminator to obtain aclassification D_(ϕ)(x) of the image (discriminating whether the imageis the generated artistic image). If the discriminator, which is oftenregarded as a powerful discriminating expert, cannot distinguish whetherthe given image is a generated one or a real artistic one, then thepurpose can be achieved that the generator is enabled to generate fakeartistic images that look like real.

The prior AI artistic creation framework is a so-called “end-to-end”process, namely, randomly inputting a vector that conforms to theGaussian distribution to directly obtain a finished artwork. Abstractimages, however, are a creation manner that expresses natural thingssubjectively through shapes and colors. An abstract image includes threeimportant components: shape, color, and an idea of reflecting thecreators' subjective thinking. The traditional “end-to-end” creationprocess of human-computer separation generates artistic images onlyrelying on Gaussian vectors generated by a system, which cannot reflectthe creators' subjective thinking, thereby failing to satisfy theconstructing components of abstract images. Thus, it is not suitable forgenerating abstract images. Besides, there is no accurate evaluationcriteria in the “end-to-end” artistic creation.

With respect to the images that are obtained only through Gaussianvectors, the qualities of the images can only be determined from acompletely subjective perspective of whether they look like artworks.Finally, the other two components of abstract images are shape andcolor. The traditional artistic creation manner regards abstract imagesas a whole to be generated, without highlighting the shape and colorcomponents. Moreover, compared with other realistic simulated artworks,changes in the shape and color components of abstract images arerelatively rich and irregular. Thus, treating abstract images as a wholeto be generated is not conducive to neural networks to learn the latentfeatures of abstract images, thereby leading to problems such as a longtraining process, an uncontrollable training result, and non-convergenceof a loss function.

SUMMARY

In view of the above-mentioned shortcomings in the prior art, thepresent invention provides a method for generating a human-computerinteractive abstract image, which can solve the problem in the prior artthat generated abstract images cannot reflect a creators' subjectivethinking, fails to display the constructing components of abstractimages, and is hard to train in the traditional approach.

In order to achieve the above object, the present invention adopts thefollowing technical solution.

A method for generating a human-computer interactive abstract imageincludes:

S1: obtaining original abstract images, and preprocessing the originalabstract images to obtain edge shape feature maps in one-to-onecorrespondence with the original abstract images; wherein the edge shapefeature maps are used as a training dataset A, and the original abstractimages are used as a training dataset B;

S2: using the training dataset A and the training dataset B as cyclegenerative objects of a Cycle-generative adversarial network (GAN)model, and training the Cycle-GAN model to capture a mappingrelationship between the edge shape feature maps and the originalabstract images;

S3: obtaining a line shape image drawn by a user; and

S4: according to the mapping relationship, intercepting a generativepart in the Cycle-GAN model that the dataset B is generated from thedataset A, discarding a cycle generative part and a discrimination partin the Cycle-GAN model, and generating a complete abstract image basedon the line shape image to generate the human-computer interactiveabstract image.

Further, step S1 includes:

S101: obtaining the original abstract images, and using the originalabstract images to construct the dataset B;

S102: performing a binarization processing on the original abstractimages in the dataset B, and extracting color edge information inbinarized images to obtain the edge shape feature maps in one-to-onecorrespondence with the original abstract images; and

S103: calculating lengths of the edge shape feature maps, and discardingedge lines with a length being greater than 150 pixels to obtain thedataset A.

Further, the Cycle-GAN model in step S2 includes a first generator G anda second generator F having an identical structure, and a firstdiscriminator D_(G) and a second discriminator D_(F) having an identicalstructure.

The first generator G is configured to capture the mapping relationshipbetween the edge shape feature maps and the original abstract images.

The second generator F is configured to capture an inverse mappingrelationship between the edge shape feature maps and the originalabstract images.

The first discriminator D_(G) is configured to discriminate a generativequality of the first generator G.

The second discriminator D_(F) is configured to discriminate agenerative quality of the second generator F.

Further, each of the first discriminator D_(G) and the seconddiscriminator D_(F) includes a first convolutional layer, a secondconvolutional layer, a third convolutional layer, a fourth convolutionallayer and a fifth convolutional layer, which are successively connected.Each of the first convolutional layer, the second convolutional layer,the third convolutional layer and the fourth convolutional layer isprovided with a normalization operation and a rectified linear unit(ReLU) function. The fifth convolutional layer is provided with aSigmoid function.

Each of the first generator G and the second generator F includes anencoding module, a residual module and a decoding module, which aresuccessively connected.

Further, the number of convolutional kernels of the first convolutionallayer is 64, a size of the convolutional kernels of the firstconvolutional layer is 4×4, and a stride of the first convolutionallayer is 2.

The number of convolutional kernels of the second convolutional layer is128, a size of the convolutional kernels of the second convolutionallayer is 4×4, and a stride of the second convolutional layer is 2.

The number of convolutional kernels of the third convolutional layer is256, a size of the convolutional kernels of the third convolutionallayer is 4×4, and a stride of the third convolutional layer is 2.

The number of convolutional kernels of the fourth convolutional layer is512, a size of the convolutional kernels of the fourth convolutionallayer is 4×4, and a stride of the fourth convolutional layer is 2.

The number of convolutional kernel of the fifth convolutional layer is1, a size of the convolutional kernel of the fifth convolutional layeris 4×4, and a stride of the fifth convolutional layer is 1.

Further, the encoding module includes a sixth convolutional layer, aseventh convolutional layer and an eighth convolutional layer, which aresuccessively connected.

Each of the sixth convolutional layer, the seventh convolutional layerand the eighth convolutional layer is provided with a normalizationoperation and a ReLU activation function.

The residual module includes a first residual layer, a second residuallayer, a third residual layer, a fourth residual layer, a fifth residuallayer and a sixth residual layer, which are successively connected. Eachof the first residual layer, the second residual layer, the thirdresidual layer, the fourth residual layer, the fifth residual layer andthe sixth residual layer is provided with a normalization operation anda ReLU activation function.

The decoding module includes a first decoding layer, a second decodinglayer and a third decoding layer, which are successively connected. Eachof the first decoding layer and the second decoding layer is providedwith a normalization layer and a ReLU activation function. The thirddecoding layer is provided with a Tanh function.

The eighth convolutional layer is connected to the first residual layer,and the sixth residual layer is connected to the first decoding layer.

Further, the number of convolutional kernels of the sixth convolutionallayer is 32, a size of the convolutional kernels of the sixthconvolutional layer is 7×7, and a stride of the sixth convolutionallayer is 1.

The number of convolutional kernels of the seventh convolutional layeris 64, a size of the convolutional kernels of the seventh convolutionallayer is 3×3, and a stride of the seventh convolutional layer is 2.

The number of convolutional kernels of the eighth convolutional layer is128, a size of the convolutional kernels of the eighth convolutionallayer is 3×3, and a stride of the eighth convolutional layer is 2.

Each of the first residual layer, the second residual layer, the thirdresidual layer, the fourth residual layer, the fifth residual layer andthe sixth residual layer includes two convolutional layers. The numberof convolutional kernels of each of the two convolutional layers is 128,a size of the convolutional kernels of each of the two convolutionallayers is 3×3, and a stride of each of the two convolutional layers is1.

The number of convolutional kernels of the first decoding layer is 64, asize of the convolutional kernels of the first decoding layer is 3×3,and a stride of the first decoding layer is 2.

The number of convolutional kernels of the second decoding layer is 32,a size of the convolutional kernels of the second decoding layer is 3×3,and a stride of the second decoding layer is 2.

The number of convolutional kernels of the third decoding layer is 3, asize of the convolutional kernels of the third decoding layer is 7×7,and a stride of the third decoding layer is 1.

Further, step S2 includes:

S201: randomly selecting a shape line image x from the dataset A as aninput of the first generator G, and obtaining a complete abstract imageŷ corresponding to the shape line image by the first generator G;

S202: using a real abstract image y in the dataset B as a positivesample, using the complete abstract image ŷ as a negative sample, andinputting the positive sample and the negative sample into the firstdiscriminator D_(G) to obtain an adversarial loss value of the firstdiscriminator D_(G);

S203: using the complete abstract image ŷ as an input of the secondgenerator F, obtaining a line shape image {circumflex over (x)}corresponding to the complete abstract image by the second generator F,and calculating a first cycle loss value according to the line shapeimage {circumflex over (x)} and the shape line image x;

S204: randomly selecting the real abstract image y from the dataset B asan input of the second generator F, and obtaining a shape line image{circumflex over (x)} corresponding to the complete real image y by thesecond generator F;

S205: using the shape line image x in the dataset A as a positivesample, using the shape line image {circumflex over (x)} obtained instep S204 as a negative sample, and inputting the positive sample andthe negative sample into the second discriminator D_(F) to obtain anadversarial loss value of the second discriminator D_(F);

S206: using the shape line image {circumflex over (x)} obtained in stepS204 as an input of the first generator G, obtaining a complete abstractimage ŷ by the first generator G, and calculating a second cycle lossvalue according to the complete abstract image ŷ and the real abstractimage y; and

S207: minimizing the adversarial loss value of the first discriminatorD_(G), the adversarial loss value of the second discriminator D_(F), thefirst cycle loss value and the second cycle loss value by using anoptimizer to complete training the Cycle-GAN model to capture themapping relationship between the edge shape feature maps and theoriginal abstract images.

Further, the adversarial loss value of the first discriminator D_(G) isexpressed as follows:

${{L_{GAN}\left( {G,D_{G}} \right)} = {\min\limits_{\Theta G}\max\limits_{\Theta D_{G}}\left\{ {{E_{y}\left\lbrack \left( {\log{D_{G}(y)}} \right. \right\rbrack} + {E_{x}\left\lbrack {\log\left( {1 - {D_{G}\left( {G(x)} \right)}} \right.} \right\rbrack}} \right\}}},$

wherein, L_(GAN)(G,D_(G)) represents the adversarial loss value of thefirst discriminator D_(G); ΘG represents a parameter of the firstgenerator G; ΘD_(G) represents a parameter of the first discriminatorD_(G); D_(G)(y) represents an output obtained by the first discriminatorD_(G) on the real abstract image y; E_(y) represents an average over allreal abstract images y; G(x) represents an output of the first generatorG for the shape line image x; D_(G)(G(x)) represents an output obtainedby the first discriminator D_(G) on a generated sample G(x); and E_(x)represents an average over all shape line images x.

The adversarial loss value of the second discriminator D_(F) isexpressed as follows:

${{L_{GAN}\left( {F,D_{F}} \right)} = {\min\limits_{\Theta F}\max\limits_{\Theta D_{F}}\left\{ {{E_{x}\left\lbrack \left( {\log{D_{F}(x)}} \right. \right\rbrack} + {E_{y}\left\lbrack {\log\left( {1 - {D_{F}\left( {F(y)} \right)}} \right.} \right\rbrack}} \right\}}},$

wherein, L_(GAN) (F,D_(F)) represents the adversarial loss value of thesecond discriminator D_(F); ΘF represents a parameter of the secondgenerator F; ΘD_(F) represents a parameter of the second discriminatorD_(F); D_(F)(y) represents an output obtained by the seconddiscriminator D_(F) on the real abstract image y; E_(y) represents theaverage over all real abstract images y; F(x) represents an output ofthe second generator F for the shape line image x; D_(F)(F(x))represents an output obtained by the second discriminator D_(F) on thegenerated sample G(x); and E_(x) represents the average over all shapeline images x.

Further, a cycle loss function of the first generator G and the secondgenerator F in step S205 is expressed as follows:L _(cyc)(D,F)=∥F(G(x))−x∥ ₁ +∥G(F(y))−y∥ ₁,

wherein, L_(cyc)(D,F) represents the cycle loss function of the firstgenerator G and the second generator F; F(G(x)) represents the lineshape image corresponding to the complete abstract image obtained by thesecond generator F; x represents the shape line image in the dataset A;G(F(y)) represents the complete abstract image obtained by the firstgenerator G; y represents the real abstract image in the dataset B.

The present invention has the following advantages.

(1) The present invention receives simple line shape images drawn byusers as inputs for generating abstract images, so that users'subjective descriptions of objective things are used to generateabstract images by the present invention. As opposed to the traditionalapproach, which uses computer-generated vectors conforming to theGaussian distribution as inputs for generating artworks, the presentinvention provides a human-computer interactive creation approachsuitable for creating abstract artworks. Besides, since the methodprovided in the present invention takes shape and color as differentcomponents of an identical abstract image to a large extent, during acreation process, color AI creation is mainly focused on, whilerecreation is performed on shape structures only based on users' inputs.This makes the difficulty of training of the generative model of thepresent invention lower than that of the traditional approach. Further,in terms of evaluation mechanism, the present invention providescreators with clearer evaluation criteria, including three dimensions ofshape, color, and idea of reflecting users' subjective thinking. Basedon the criteria, users can more objectively describe the qualities ofgenerated images.

(2) Different from the traditional approach, the present inventionrealizes the separation of the shape and color components of theabstract images. Besides, the key point enabling the generated artworkto reflect users' subjective thinking in the human-computer interactivecreation is to extract shape features of the artwork from the data ofthe abstract image by the edge feature extracting operator, therebyestablishing a matching relationship between the shape and the color ofthe abstract image.

(3) The present invention provides a method for generating ahuman-computer interactive abstract image based on a Cycle-GAN. Simpleline shape images drawn by users are input, and corresponding finishedabstract images are generated.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of a traditional approach in the priorart.

FIG. 2 is a schematic diagram of edge features of the abstract imageshape component extraction in an embodiment.

FIG. 3 is a flow chart of the method of the present invention.

FIG. 4 is a schematic diagram of an overall structure of a Cycle-GANaccording to the present invention.

FIG. 5 is a schematic diagram of examples generated according to anembodiment of the present invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

The specific embodiments of the present invention are described below tofacilitate those skilled in the art to understand the present invention.However, it should be noted that the present invention is not limited tothe scope of the specific embodiments. For those skilled in the art, aslong as various changes are within the spirit and scope of the presentinvention defined and discriminated by the appended claims, thesechanges are obvious, and all solutions that are made taking advantage ofthe present invention shall fall within the scope of protection of thepresent invention.

Embodiment

The present invention provides a method for generating a human-computerinteractive abstract image based on an edge feature extracting operatorand a Cycle-GAN. The present invention allows users to first observe anobjective thing and draw a simple line shape that is related to theobjective thing but is created subjectively. The present inventionreceives the shape lines drawn by users, and, on this basis, performsrecreation on the shape and complete AI creation on colors, and finallyoutputs an artwork of an abstract image that contains a shape, a colorand an idea capable of reflecting the users' subjective thinking. Asshown in FIG. 2 , different from the traditional approach, the presentinvention realizes the separation of the shape and color components ofthe abstract images. Besides, the key point enabling the generatedartwork to reflect the users' subjective thinking in the human-computerinteractive creation is to extract shape features of the artwork fromthe data of the abstract image by the edge feature extracting operator,thereby establishing a matching relationship between the shape and thecolor of the abstract image. As shown in FIG. 3 , in an embodiment, themethod provided in the present invention includes the following steps.

S1: original abstract images are obtained, and the original abstractimages are preprocessed to obtain edge shape feature maps in one-to-onecorrespondence with the original abstract images. The edge shape featuremaps are used as the training dataset A, and the original abstractimages are used as the training dataset B.

In an embodiment, step S1 includes:

S101: the original abstract images are obtained, and the originalabstract images are used to construct the dataset B;

S102: binarization processing is performed on the original abstractimages in the dataset B, and color edge information in binarized imagesis extracted to obtain the edge shape feature maps in one-to-onecorrespondence with the original abstract images; and

S103: lengths of the edge shape feature maps are calculated, and edgelines with a length being greater than 150 pixels are discarded toobtain the dataset A.

In an embodiment, the present invention selects abstract images fromWiKiArt and other related websites. A total of 4415 images are used toconstruct the dataset B of the original abstract images for training.The OpenCV2 image processing tool is used to process each of theoriginal abstract images in the dataset B. First, binarizationprocessing is performed on the images using a threshold function, wherea binarization threshold is an average of colors of a current image.Then, the color edge information in the binarized images is extracted byusing a findContours function. However, since not all changes in thecolors indicate meaningful shape structures, there is a need to limitthe length of extracted edge information. The present inventioncalculates the length of each edge line by using an arcLength function,and sets a length discarding threshold to be 150 pixels, so as to obtainthe training dataset A with the same amount of elements as the datasetB.

S2: the training dataset A and the training dataset B are used as cyclegenerative objects of a Cycle-GAN model, and the Cycle-GAN model istrained to capture a mapping relationship between the edge shape featuremaps and the original abstract images.

In an embodiment, step S2 includes:

S201: a shape line image x is randomly selected from the dataset A as aninput of the first generator G, and a complete abstract image ŷcorresponding to the shape line image is obtained by the first generatorG;

S202: a real abstract image y in the dataset B is used as a positivesample, the complete abstract image ŷ is used as a negative sample, andthe positive sample and the negative sample are input into the firstdiscriminator D_(G) to obtain an adversarial loss value of the firstdiscriminator D_(G);

S203: the complete abstract image ŷ is used as an input of the secondgenerator F, a line shape image {circumflex over (x)} corresponding tothe complete abstract image is obtained by the second generator F, and afirst cycle loss value is calculated according to the line shape image{circumflex over (x)} and the shape line image x;

S204: the real abstract image y is randomly selected from the dataset Bas an input of the second generator F, and a shape line image{circumflex over (x)} corresponding to the real abstract image y isobtained by the second generator F;

S205: the shape line image x in the dataset A is used as a positivesample, the shape line image {circumflex over (x)} obtained in step S204is used as a negative sample, and the positive sample and the negativesample are input into the second discriminator D_(F) to obtain anadversarial loss value of the second discriminator D_(F);

S206: the shape line image {circumflex over (x)} obtained in step S204is used as an input of the first generator G, a complete abstract imageŷ is obtained by the first generator G, and a second cycle loss value iscalculated according to the complete abstract image ŷ and the realabstract image y; and

S207: the adversarial loss value of the first discriminator D_(G), theadversarial loss value of the second discriminator D_(F), the firstcycle loss value and the second cycle loss value are minimized by usingan optimizer to complete training the Cycle-GAN model to capture themapping relationship between the edge shape feature maps and theoriginal abstract images.

In an embodiment, the adversarial loss value of the first discriminatorD_(G) is expressed as follows:

${{L_{GAN}\left( {G,D_{G}} \right)} = {\min\limits_{\Theta G}\max\limits_{\Theta D_{G}}\left\{ {{E_{y}\left\lbrack \left( {\log{D_{G}(y)}} \right. \right\rbrack} + {E_{x}\left\lbrack {\log\left( {1 - {D_{G}\left( {G(x)} \right)}} \right.} \right\rbrack}} \right\}}},$

where, L_(GAN) (G,D_(G)) represents the adversarial loss value of thefirst discriminator D_(G); ΘG represents a parameter of the firstgenerator G; ΘD_(G) represents a parameter of the first discriminatorD_(G); D_(G)(y) represents an output obtained by the first discriminatorD_(G) on the real abstract image y; E_(y) represents an average over allreal abstract images y; G(x) represents an output of the first generatorG for the shape line image x; D_(G)(G(x)) represents an output obtainedby the first discriminator D_(G) on a generated sample G(x); and E_(x)represents an average over all shape line images x.

The adversarial loss value of the second discriminator D_(F) isexpressed as follows:

${{L_{GAN}\left( {F,D_{F}} \right)} = {\min\limits_{\Theta F}\max\limits_{\Theta D_{F}}\left\{ {{E_{x}\left\lbrack \left( {\log{D_{F}(x)}} \right. \right\rbrack} + {E_{y}\left\lbrack {\log\left( {1 - {D_{F}\left( {F(y)} \right)}} \right.} \right\rbrack}} \right\}}},$

where, L_(GAN)(F,D_(F)) represents the adversarial loss value of thesecond discriminator D_(F); ΘF represents a parameter of the secondgenerator F; ΘD_(F) represents a parameter of the second discriminatorD_(F); D_(F)(y) represents an output obtained by the seconddiscriminator D_(F) on the real abstract image y; E_(y) represents theaverage over all real abstract images y; F(x) represents an output ofthe second generator F for the shape line image x; D_(F)(F(x))represents an output obtained by the second discriminator D_(F) on thegenerated sample G(x); and E_(x) represents the average over all shapeline images x.

In an embodiment, with respect to a traditional adversarial lossfunction, the Cycle-GAN model further introduces a cycle loss functionto ensure the cycle consistency of generated images. The cycle lossfunction of the first generator G and the second generator F isexpressed as follows:L _(cyc)(D,F)=∥F(G(x))−x∥ ₁ +∥G(F(y))−y∥ ₁,

where, L_(cyc)(D,F) represents the cycle loss function of the firstgenerator G and the second generator F; F(G(x)) represents the lineshape image corresponding to the complete abstract image obtained by thesecond generator F; x represents the shape line image in the dataset A;G(F(y)) represents the complete abstract image obtained by the firstgenerator G; y represents the real abstract image in the dataset B.

In an embodiment, the cycle loss function ensures that the inversemapping F can map the complete abstract image generated by the mapping Gback to the structural information, and also ensures that the mapping Gcan map the structural information generated by the inverse mapping Fback to the real abstract image, so as to further ensure the reality ofthe complete abstract image generated by the mapping G.

A loss function of the entire training process is:L=L _(GAN)(G,D _(G))+L _(GAN)(F,D _(F))+L _(cyc)(G,F)

In an embodiment, the present invention uses the Cycle-GAN model tolearn the mutual mapping relationship between the dataset A and thedataset B, namely learning the mapping G: A→B from a simple structure toa complete abstract image and the inverse mapping F: B→A from thecomplete abstract image to the simple structure. Although the presentinvention does not use the inverse mapping F to generate an abstractimage, the inverse mapping F provides a cycle consistency for theCycle-GAN.

In an embodiment, the mapping G and the inverse mapping F are generatorsin the Cycle-GAN. The generators (namely the first generator G and thesecond generator F) are provided with discriminators to supervise thegenerative qualities of the generators, which are the firstdiscriminator D_(G) and the second discriminator D_(F), respectively.Each of the discriminators is a 70×70 PatchGAN discriminator. Theoverall structure of the Cycle-GAN is shown in FIG. 4 . In theCycle-GAN, the first generator G and the second generator F are codecsand have an identical structure. The details are as follows:

The Cycle-GAN model includes a first generator G and a second generatorF having an identical structure, and a first discriminator D_(G) and asecond discriminator D_(F) having an identical structure. The firstgenerator G is configured to capture a mapping relationship between theedge shape feature maps and the original abstract images. The secondgenerator F is configured to capture an inverse mapping relationshipbetween the edge shape feature maps and the original abstract images.The first discriminator D_(G) is configured to discriminate a generativequality of the first generator G. The second discriminator D_(F) isconfigured to discriminate a generative quality of the second generatorF. Each of the first discriminator D_(G) and the second discriminatorD_(F) includes a first convolutional layer, a second convolutionallayer, a third convolutional layer, a fourth convolutional layer and afifth convolutional layer, which are successively connected. Each of thefirst convolutional layer, the second convolutional layer, the thirdconvolutional layer and the fourth convolutional layer is provided witha normalization operation and a rectified linear unit (ReLU) function.The fifth convolutional layer is provided with a Sigmoid function. Eachof the first generator G and the second generator F includes an encodingmodule, a residual module and a decoding module, which are successivelyconnected. As shown in Table 1, Table 1 is a table of the structure ofthe first discriminator D_(G) and the second discriminator D_(F). Thenumber of convolutional kernels of the first convolutional layer is 64,the size of the convolutional kernels of the first convolutional layeris 4×4, and the stride of the first convolutional layer is 2. The numberof convolutional kernels of the second convolutional layer is 128, thesize of the convolutional kernels of the second convolutional layer is4×4, and the stride of the second convolutional layer is 2. The numberof convolutional kernels of the third convolutional layer is 256, thesize of the convolutional kernels of the third convolutional layer is4×4, and the stride of the third convolutional layer is 2. The number ofconvolutional kernels of the fourth convolutional layer is 512, the sizeof the convolutional kernels of the fourth convolutional layer is 4×4,and the stride of the fourth convolutional layer is 2. The number ofconvolutional kernel of the fifth convolutional layer is 1, the size ofthe convolutional kernel of the fifth convolutional layer is 4×4, andthe stride of the fifth convolutional layer is 1. As shown in Table 2,Table 2 is a table of the structure of the first generator G and thesecond generator F. The encoding module includes a sixth convolutionallayer, a seventh convolutional layer and an eighth convolutional layer,which are successively connected. Each of the sixth convolutional layer,the seventh convolutional layer and the eighth convolutional layer isprovided with a normalization operation and a ReLU activation function.The residual module includes a first residual layer, a second residuallayer, a third residual layer, a fourth residual layer, a fifth residuallayer and a sixth residual layer, which are successively connected. Eachof the first residual layer, the second residual layer, the thirdresidual layer, the fourth residual layer, the fifth residual layer andthe sixth residual layer is provided with a normalization operation anda ReLU activation function. The decoding module includes a firstdecoding layer, a second decoding layer and a third decoding layer,which are successively connected. Each of the first decoding layer andthe second decoding layer is provided with a normalization layer and aReLU activation function. The third decoding layer is provided with aTanh function. The eighth convolutional layer is connected to the firstresidual layer, and the sixth residual layer is connected to the firstdecoding layer. The number of convolutional kernels of the sixthconvolutional layer is 32, the size of the convolutional kernels of thesixth convolutional layer is 7×7, and the stride of the sixthconvolutional layer is 1. The number of convolutional kernels of theseventh convolutional layer is 64, the size of the convolutional kernelsof the seventh convolutional layer is 3×3, and the stride of the seventhconvolutional layer is 2. The number of convolutional kernels of theeighth convolutional layer is 128, the size of the convolutional kernelsof the eighth convolutional layer is 3×3, and the stride of the eighthconvolutional layer is 2. Each of the first residual layer, the secondresidual layer, the third residual layer, the fourth residual layer, thefifth residual layer and the sixth residual layer includes twoconvolutional layers. The number of convolutional kernels of each of thetwo convolutional layers is 128, the size of the convolutional kernelsof each of the two convolutional layers is 3×3, and the stride of eachof the two convolutional layers is 1. The number of convolutionalkernels of the first decoding layer is 64, the size of the convolutionalkernels of the first decoding layer is 3×3, and the stride of the firstdecoding layer is 2. The number of convolutional kernels of the seconddecoding layer is 32, the size of the convolutional kernels of thesecond decoding layer is 3×3, and the stride of the second decodinglayer is 2. The number of convolutional kernels of the third decodinglayer is 3, the size of the convolutional kernels of the third decodinglayer is 7×7, and the stride of the third decoding layer is 1.

TABLE 1 Number of Size of Name of Convo- Convo- Output Convolutionallutional lutional Normalization Activation Layer Kernels Kernels StrideOperation Function first 64 4 × 4 2 batch ReLU convolutionalnormalization layer second 128 4 × 4 2 batch ReLU convolutionalnormalization layer third 256 4 × 4 2 batch ReLU convolutionalnormalization layer fourth 512 4 × 4 2 batch ReLU convolutionalnormalization layer fifth 1 4 × 4 1 Sigmoid convolutional layer

TABLE 2 Number Size of Output Activ- Compo- of Convo- Convo- Normal-ation Module nent lutional lutional ization Func- Name Name KernelsKernels Stride Operation tion encoding sixth 32 7 × 7 1 single ReLUlayer convo- instance lutional normal- layer ization seventh 64 3 × 3 2single ReLU convo- instance lutional normal- layer ization eighth 128 3× 3 2 single ReLU convo- instance lutional normal- layer izationresidual first 128 3 × 3 1 single ReLU module residual instance layernormal- ization 128 3 × 3 1 single ReLU instance standard- izationsecond 128 3 × 3 1 single ReLU residual instance layer normal- ization128 3 × 3 1 single ReLU instance normal- ization third 128 3 × 3 1single ReLU residual instance layer normal- ization 128 3 × 3 1 singleReLU instance normal- ization fourth 128 3 × 3 1 single ReLU residualinstance layer normal- ization 128 3 × 3 1 single ReLU instance normal-ization fifth 128 3 × 3 1 single ReLU residual instance layer normal-ization 128 3 × 3 1 single ReLU instance normal- ization sixth 128 3 × 31 single ReLU residual instance layer normal- ization 128 3 × 3 1 singleReLU instance normal- ization decoding first 64 3 × 3 2 single ReLUmodule decoding instance layer normal- ization second 32 3 × 3 2 singleReLU decoding instance layer normal- ization third 3 7 × 7 1 Tanhdecoding layer

S3: a line shape image drawn by a user is obtained.

In an embodiment, drawing panels are provided for users to receiveusers' actions of drawing lines to then form line shape images. Thereare various types of such drawing panels. A simple manner is that usersuse drawing tools of tablet computers to draw image files.

S4: according to the mapping relationship, a generative part in theCycle-GAN model that the dataset B is generated from the dataset A isintercepted, a cycle generative part and a discrimination part in theCycle-GAN model are discarded, and a complete abstract image isgenerated based on the line shape image to generate the human-computerinteractive abstract image.

In an embodiment, assuming a generation process a→b→a′, where arepresents a real shape line image, b represents a generated completeabstract image, and a′ represents a generated shape line image, thegenerative part refers to the part a→b, the cycle generative part refersto the part b→a′, and the discrimination part refers to discriminating,by a discriminator, whether b′ is generated. During the trainingprocess, all the three parts are needed; while after the training, onlythe part a→b is needed.

In an embodiment, the first generator G and the second generator F areobtained. The present invention involves a process of generatingcomplete abstract images from shape line images, thus the firstgenerator G is selected as the final generative model, receive users'input images at the same time, and outputs corresponding generatedabstract images.

In an embodiment, since the present invention is based on line shapeimages subjectively drawn by users, in addition to relying on thecompletely subjective criterion of “looks like abstract images”, userscan also determine the quality of the generated abstract images from thefollowing aspects:

(1) Whether the generated abstract images reflect line shape featuresinput by users. Users' inputs are not only shape features, but also animportant basis for abstract images generated by the method according tothe present invention to reflect users' subjective thinking. Thus, thegenerated abstract images need to clearly reflect users' input shapes.

(2) Whether the generated abstract images are recreated on line shapesinput by users. Since users are non-professional painters, line shapesinput by them may be excessively simple or have insufficient structures.The adversarial loss function of the Cycle-GAN model ensures that thegenerated abstract images are as consistent as possible with abstractimages drawn by professional painters. This means that the presentinvention needs to perform recreation on the non-professional shapestructures input by users to obtain more complete abstract structures.

(3) Whether complete AI color creation is performed on the generatedabstract images. The present invention separates a shape and a color ofabstract images by the Cycle-GAN model. After users input a basic shapeimage, the present invention should return a complete abstract imagecontaining both the shape and the color. This means that the computerneeds to perform independent and complete AI color creation.

As shown in FIG. 5 , three examples given in FIG. 5 show generatedresults according to the present invention. In the first group ofsunflowers, firstly, the generated result clearly reflects the inputshape. Secondly, a shape structure that wraps the lines is formed inaddition to the main part of the sunflowers, which performs therecreation on the basis of a given shape. Finally, independent andcomplete AI creation is made for the colors of the flower petals andflower cores of the sunflowers, as well as the colors of differentbackgrounds. Besides, the flower cores of the sunflowers using brightred instead of single brown can reflect the abstract artistic expressionform.

What is claimed is:
 1. A method for generating a human-computerinteractive abstract image, comprising the steps of: S1: obtainingoriginal abstract images, and preprocessing the original abstract imagesto obtain edge shape feature maps in one-to-one correspondence with theoriginal abstract images; wherein the edge shape feature maps are usedas a training dataset A, and the original abstract images are used as atraining dataset B; S2: using the training dataset A and the trainingdataset B as cycle generative objects of a Cycle-generative adversarialnetwork (GAN) model, and training the Cycle-GAN model to capture amapping relationship between the edge shape feature maps and theoriginal abstract images; S3: obtaining a line shape image drawn by auser; and S4: according to the mapping relationship, intercepting agenerative part in the Cycle-GAN model that the training dataset B isgenerated from the training dataset A, discarding a cycle generativepart and a discrimination part in the Cycle-GAN model, and generating acomplete abstract image based on the line shape image to generate thehuman-computer interactive abstract image.
 2. The method according toclaim 1, wherein step S1 comprises: S101: obtaining the originalabstract images, and using the original abstract images to construct thetraining dataset B; S102: performing a binarization processing on theoriginal abstract images in the training dataset B to obtain binarizedimages, and extracting color edge information in the binarized images toobtain the edge shape feature maps in one-to-one correspondence with theoriginal abstract images; and S103: calculating lengths of edge lines ofthe edge shape feature maps, and discarding edge lines with a lengthbeing greater than 150 pixels to obtain the training dataset A.
 3. Themethod according to claim 1, wherein the Cycle-GAN model in step S2comprises a first generator G, a second generator F, a firstdiscriminator D_(G) and a second discriminator D_(F); wherein the firstgenerator G and the second generator F are identical structurally, andthe first discriminator D_(G) and the second discriminator D_(F) areidentical structurally; the first generator G is configured to capturethe mapping relationship between the edge shape feature maps and theoriginal abstract images; the second generator F is configured tocapture an inverse mapping relationship between the edge shape featuremaps and the original abstract images; the first discriminator D_(G) isconfigured to discriminate a generative quality of the first generatorG; and the second discriminator D_(F) is configured to discriminate agenerative quality of the second generator F.
 4. The method according toclaim 3, wherein each of the first discriminator D_(G) and the seconddiscriminator D_(F) comprises a first convolutional layer, a secondconvolutional layer, a third convolutional layer, a fourth convolutionallayer and a fifth convolutional layer, wherein the first convolutionallayer, the second convolutional layer, the third convolutional layer,the fourth convolutional layer and the fifth convolutional layer aresuccessively connected; each of the first convolutional layer, thesecond convolutional layer, the third convolutional layer and the fourthconvolutional layer is provided with a first normalization operation anda rectified linear unit (ReLU) activation function; the fifthconvolutional layer is provided with a Sigmoid function; and each of thefirst generator G and the second generator F comprises an encodingmodule, a residual module and a decoding module, wherein the encodingmodule, the residual module and the decoding module are successivelyconnected.
 5. The method according to claim 4, wherein a number ofconvolutional kernels of the first convolutional layer is 64, a size ofthe convolutional kernels of the first convolutional layer is 4×4, and astride of the first convolutional layer is 2; a number of convolutionalkernels of the second convolutional layer is 128, a size of theconvolutional kernels of the second convolutional layer is 4×4, and astride of the second convolutional layer is 2; a number of convolutionalkernels of the third convolutional layer is 256, a size of theconvolutional kernels of the third convolutional layer is 4×4, and astride of the third convolutional layer is 2; a number of convolutionalkernels of the fourth convolutional layer is 512, a size of theconvolutional kernels of the fourth convolutional layer is 4×4, and astride of the fourth convolutional layer is 2; and a number ofconvolutional kernel of the fifth convolutional layer is 1, a size ofthe convolutional kernel of the fifth convolutional layer is 4×4, and astride of the fifth convolutional layer is
 1. 6. The method according toclaim 5, wherein the encoding module comprises a sixth convolutionallayer, a seventh convolutional layer and an eighth convolutional layer,wherein the sixth convolutional layer, the seventh convolutional layerand the eighth convolutional layer are successively connected; each ofthe sixth convolutional layer, the seventh convolutional layer and theeighth convolutional layer is provided with a second normalizationoperation and the ReLU activation function; the residual modulecomprises a first residual layer, a second residual layer, a thirdresidual layer, a fourth residual layer, a fifth residual layer and asixth residual layer, wherein the first residual layer, the secondresidual layer, the third residual layer, the fourth residual layer, thefifth residual layer and the sixth residual layer are successivelyconnected; each of the first residual layer, the second residual layer,the third residual layer, the fourth residual layer, the fifth residuallayer and the sixth residual layer is provided with the secondnormalization operation and the ReLU activation function; the decodingmodule comprises a first decoding layer, a second decoding layer and athird decoding layer, wherein the first decoding layer, the seconddecoding layer and the third decoding layer are successively connected;each of the first decoding layer and the second decoding layer isprovided with the second normalization operation and the ReLU activationfunction; the third decoding layer is provided with a Tanh function; andthe eighth convolutional layer is connected to the first residual layer,and the sixth residual layer is connected to the first decoding layer.7. The method according to claim 6, wherein a number of convolutionalkernels of the sixth convolutional layer is 32, a size of theconvolutional kernels of the sixth convolutional layer is 7×7, and astride of the sixth convolutional layer is 1; a number of convolutionalkernels of the seventh convolutional layer is 64, a size of theconvolutional kernels of the seventh convolutional layer is 3×3, and astride of the seventh convolutional layer is 2; a number ofconvolutional kernels of the eighth convolutional layer is 128, a sizeof the convolutional kernels of the eighth convolutional layer is 3×3,and a stride of the eighth convolutional layer is 2; each of the firstresidual layer, the second residual layer, the third residual layer, thefourth residual layer, the fifth residual layer and the sixth residuallayer comprises two convolutional layers; a number of convolutionalkernels of each of the two convolutional layers is 128, a size of theconvolutional kernels of each of the two convolutional layers is 3×3,and a stride of each of the two convolutional layers is 1; a number ofconvolutional kernels of the first decoding layer is 64, a size of theconvolutional kernels of the first decoding layer is 3×3, and a strideof the first decoding layer is 2; a number of convolutional kernels ofthe second decoding layer is 32, a size of the convolutional kernels ofthe second decoding layer is 3×3, and a stride of the second decodinglayer is 2; and a number of convolutional kernels of the third decodinglayer is 3, a size of the convolutional kernels of the third decodinglayer is 7×7, and a stride of the third decoding layer is
 1. 8. Themethod according to claim 7, wherein step S2 comprises: S201: randomlyselecting a first shape line image x from the training dataset A as afirst input of the first generator G, and obtaining a first completeabstract image ŷ corresponding to the first shape line image x by thefirst generator G; S202: using a real abstract image y in the trainingdataset B as a first positive sample, using the first complete abstractimage ŷ as a first negative sample, and inputting the first positivesample and the first negative sample into the first discriminator D_(G)to obtain an adversarial loss value of the first discriminator D_(G);S203: using the first complete abstract image ŷ as a first input of thesecond generator F, obtaining a line shape image {circumflex over (x)}corresponding to the first complete abstract image ŷ by the secondgenerator F, and calculating a first cycle loss value according to theline shape image {circumflex over (x)} and the first shape line image x;S204: randomly selecting the real abstract image y from the trainingdataset B as a second input of the second generator F, and obtaining asecond shape line image {circumflex over (x)} corresponding to the realabstract image y by the second generator F; S205: using the first shapeline image x in the training dataset A as a second positive sample,using the second shape line image {circumflex over (x)} obtained in stepS204 as a second negative sample, and inputting the second positivesample and the second negative sample into the second discriminatorD_(F) to obtain an adversarial loss value of the second discriminatorD_(F); S206: using the second shape line image {circumflex over (x)}obtained in step S204 as a second input of the first generator G,obtaining a second complete abstract image ŷ by the first generator G,and calculating a second cycle loss value according to the secondcomplete abstract image ŷ and the real abstract image y; and S207:minimizing the adversarial loss value of the first discriminator D_(G),the adversarial loss value of the second discriminator D_(F), the firstcycle loss value and the second cycle loss value by using an optimizerto complete training the Cycle-GAN model to capture the mappingrelationship between the edge shape feature maps and the originalabstract images.
 9. The method according to claim 8, wherein theadversarial loss value of the first discriminator D_(G) is expressed asfollows:${{L_{GAN}\left( {G,D_{G}} \right)} = {\min\limits_{\Theta G}\max\limits_{\Theta D_{G}}\left\{ {{E_{y}\left\lbrack \left( {\log{D_{G}(y)}} \right. \right\rbrack} + {E_{x}\left\lbrack {\log\left( {1 - {D_{G}\left( {G(x)} \right)}} \right.} \right\rbrack}} \right\}}},$wherein, L_(GAN)(G,D_(G)) represents the adversarial loss value of thefirst discriminator D_(G); ΘG represents a parameter of the firstgenerator G; ΘD_(G) represents a parameter of the first discriminatorD_(G); D_(G)(y) represents a first output obtained by the firstdiscriminator D_(G) on the real abstract image y; E_(y) represents anaverage over all real abstract images y; G(x) represents an output ofthe first generator G for the first shape line image x; D_(G)(G(x))represents a second output obtained by the first discriminator D_(G) ona generated sample G(x); and E_(x) represents an average over all firstshape line images x; the adversarial loss value of the seconddiscriminator D_(F) is expressed as follows:${{L_{GAN}\left( {F,D_{F}} \right)} = {\min\limits_{\Theta F}\max\limits_{\Theta D_{F}}\left\{ {{E_{x}\left\lbrack \left( {\log{D_{F}(x)}} \right. \right\rbrack} + {E_{y}\left\lbrack {\log\left( {1 - {D_{F}\left( {F(y)} \right)}} \right.} \right\rbrack}} \right\}}},$wherein, L_(GAN)(F,D_(F)) represents the adversarial loss value of thesecond discriminator D_(F); ΘF represents a parameter of the secondgenerator F; ΘD_(F) represents a parameter of the second discriminatorD_(F); D_(F)(y) represents a third output obtained by the seconddiscriminator D_(F) on the real abstract image y; E_(y) represents theaverage over all real abstract images y; F(x) represents an output ofthe second generator F for the first shape line image x; D_(F)(F(x))represents a fourth output obtained by the second discriminator D_(F) onthe generated sample G(x); and E_(x) represents the average over allfirst shape line images x.
 10. The method according to claim 8, whereina cycle loss function of the first generator G and the second generatorF in step S205 is expressed as follows:L _(cyc)(D,F)=∥F(G(x))−x∥ ₁ +G(F(y))−y∥ ₁, wherein, L_(cyc)(D,F)represents the cycle loss function of the first generator G and thesecond generator F; F(G(x)) represents the line shape imagecorresponding to the complete abstract image obtained by the secondgenerator F; x represents the first shape line image in the trainingdataset A; G(F(y)) represents the second complete abstract imageobtained by the first generator G; y represents the real abstract imagein the training dataset B.